Failed Interop Tests

QUIC Interop Runner, client vs. server

• aioquic vs. neqo-latest: R A
• go-x-net vs. neqo-latest: A
• kwik vs. neqo-latest: A
• lsquic vs. neqo-latest: R A
• msquic vs. neqo-latest: Z A
• mvfst vs. neqo-latest: Z 3 A L1 C1
• neqo vs. neqo-latest: Z A
• neqo-latest vs. aioquic: Z C1
• neqo-latest vs. haproxy: Z
• neqo-latest vs. lsquic: Z
• neqo-latest vs. msquic: Z A
• neqo-latest vs. mvfst: DC U A L1 L2 C1 C2
• neqo-latest vs. neqo: Z A
• neqo-latest vs. neqo-latest: A L1
• neqo-latest vs. ngtcp2: Z
• neqo-latest vs. quic-go: Z C1
• neqo-latest vs. quiche: L1 C1
• neqo-latest vs. quinn: E A
• neqo-latest vs. s2n-quic: R
• neqo-latest vs. xquic: Z A C1
• ngtcp2 vs. neqo-latest: A
• picoquic vs. neqo-latest: R A
• quic-go vs. neqo-latest: A
• quiche vs. neqo-latest: R 3 A
• quinn vs. neqo-latest: R Z E A
• s2n-quic vs. neqo-latest: E A
• xquic vs. neqo-latest: M A

Codecov Report

Attention: Patch coverage is 75.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 93.20%. Comparing base (3015218) to head (beb1c2d).

@@           Coverage Diff           @@
##             main    #1895   +/-   ##
=======================================
  Coverage   93.20%   93.20%           
=======================================
  Files         111      111           
  Lines       36027    36028    +1     
=======================================
+ Hits        33579    33580    +1     
  Misses       2448     2448           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Benchmark results

Performance differences relative to 3015218.

• drain a timer quickly time: [319.27 ns 326.27 ns 332.84 ns]
  change: [+2.1180% +4.1448% +6.1334%] (p = 0.00 < 0.05)
  💔 Performance has regressed.

• coalesce_acked_from_zero 1+1 entries
  time: [194.87 ns 195.27 ns 195.72 ns]
  change: [+0.2384% +0.7057% +1.2693%] (p = 0.00 < 0.05)
  Change within noise threshold.

• coalesce_acked_from_zero 3+1 entries
  time: [237.45 ns 238.10 ns 238.73 ns]
  change: [-1.6690% -1.2327% -0.7340%] (p = 0.00 < 0.05)
  Change within noise threshold.

• coalesce_acked_from_zero 10+1 entries
  time: [237.07 ns 237.68 ns 238.46 ns]
  change: [-2.1043% -1.3798% -0.7122%] (p = 0.00 < 0.05)
  Change within noise threshold.

• coalesce_acked_from_zero 1000+1 entries
  time: [218.29 ns 226.39 ns 244.40 ns]
  change: [-0.3824% +1.7817% +6.6591%] (p = 0.40 > 0.05)
  No change in performance detected.

• RxStreamOrderer::inbound_frame()
  time: [119.95 ms 120.00 ms 120.06 ms]
  change: [+1.0481% +1.1291% +1.2110%] (p = 0.00 < 0.05)
  💔 Performance has regressed.

• transfer/Run multiple transfers with varying seeds
  time: [120.86 ms 121.15 ms 121.44 ms]
  thrpt: [32.939 MiB/s 33.018 MiB/s 33.097 MiB/s]
  change:
  time: [+0.2904% +0.6373% +0.9771%] (p = 0.00 < 0.05)
  thrpt: [-0.9677% -0.6333% -0.2895%]
  Change within noise threshold.

• transfer/Run multiple transfers with the same seed
  time: [120.92 ms 121.18 ms 121.46 ms]
  thrpt: [32.933 MiB/s 33.008 MiB/s 33.081 MiB/s]
  change:
  time: [+0.2614% +0.5737% +0.8999%] (p = 0.00 < 0.05)
  thrpt: [-0.8918% -0.5704% -0.2607%]
  Change within noise threshold.

• 1-conn/1-100mb-resp (aka. Download)/client
  time: [1.1467 s 1.1509 s 1.1557 s]
  thrpt: [86.527 MiB/s 86.886 MiB/s 87.206 MiB/s]
  change:
  time: [+0.4953% +0.9694% +1.4900%] (p = 0.00 < 0.05)
  thrpt: [-1.4682% -0.9601% -0.4928%]
  Change within noise threshold.

• 1-conn/10_000-parallel-1b-resp (aka. RPS)/client
  time: [428.68 ms 430.82 ms 432.99 ms]
  thrpt: [23.095 Kelem/s 23.211 Kelem/s 23.327 Kelem/s]
  change:
  time: [-0.5555% +0.1291% +0.8129%] (p = 0.72 > 0.05)
  thrpt: [-0.8063% -0.1290% +0.5586%]
  No change in performance detected.

• 1-conn/1-1b-resp (aka. HPS)/client
  time: [49.596 ms 49.845 ms 50.121 ms]
  thrpt: [19.952 elem/s 20.062 elem/s 20.163 elem/s]
  change:
  time: [-0.5082% +0.0951% +0.6163%] (p = 0.76 > 0.05)
  thrpt: [-0.6125% -0.0950% +0.5108%]
  No change in performance detected.

Client/server transfer results

Transfer of 134217728 bytes over loopback.

⬇️ Download logs

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.

• Linux: Debug Release
• macOS: Debug Release
• Windows: Debug Release